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DETAILED ACTION 

This is a Non-Final Office Action in response to the RCE/amendments/remarks 
filed 06/03/2009. The current patent application originally filed 03/30/2004. (Assignee: 
Ricoh). 

• Claims 1 -9; 1 1 ; 1 3-27; 29-39; 41 -47; 49-56; 58-64 and 66-69 are pending. 

• Claims 1 ; 21 ; 34; 44; 51 ; 61 ; 68-69 (currently amended) and are 
independent claims. 

• Claims 10; 28; 40; 48; 57 and 6 were previously canceled. 



Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
06/03/2009 has been entered. 
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Claims Rejection - 35 U.S.C. 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

Claims 1-9; 11; 13-27; 29-39; 41-47; 49-56; 58-64 and 66-69 are rejected under 
35 U.S.C. 103(a) as being as being unpatentable over Chiu et al US007051271 B1 filed 
05/31/2000 (hereinafter Chiu), in view of Coar US 200701 06932A1- Continuation of 
09/533,152 filed 03/23/2000 (hereinafter Coar). 

Regarding independent claims 1, 

Chiu teaches: 

a method in a computer system for creating a composite electronic 
representation including presentation material information, the computer 
system thus creating a composite electronic representation of the 
document, 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video). This interpretation is supported by Applicant's specification, which 
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is stated "creating a composite electronic representation comprising the determined 
information" at Pages 3 Para 15. ) 

the method comprising: scanning a paper document to generate an 
electronic presentation of the document with presentation material. 

(See the Abstract and Col. 2, Lines 15-25^Chiu discloses this limitation in that a 
computer system for creating a composite electronic representation including 
presentation material information (i.e. annotations made on the paper documents during 
the meeting can be extracted and used as indexes to the video.) in addition Chiu further 
discloses the a scanned document is linking to a segment of a video wherein the 
scanned document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step includes an identifier for comparing to the plurality of 
video frame identifiers; 

This interpretation is supported by Applicant's Specification, which states "key 
frame images obtained from video information captured during the presentation" at 
Pages 21 Para 57. ) 

extracting a visual feature from the electronic presentation of the 
document, the visual feature corresponding to a portion of the presentation 
material; 

(See Chiu at the Abstract, teaches annotations made on the paper documents during 
the meeting can be extracted and used as indexes to the video. An orthonormal 
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transform, such as a Digital Cosine Transform (DCT) is used to compare scanned 
documents to video frames.) 

accessing recorded information including at least one of audio and 
visual information recorded during a presentation of the presentation 
material; 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 
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storing the composite electronic representation for access by the 
user or another user accessing the composite electronic document. 

(See Chui at Fig. 5 and Col. 8 Line 5 discloses memory item 806 and persistent storage 
808 stores a scanned digital document 101a and video file 200.) 

Also see Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames responsive to 
the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a. 

Also see Chiu at Fig. 5 and Col. 8 Line 5 discloses memory item 806 and 
persistent storage 808 stores a scanned digital document 101a and video file 200.) 
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comparing the visual feature to the recorded information to 
determine a portion of the recorded information that matches the visual 
feature, whereby at least a portion of the recorded information matches a 
feature portion of the presentation material, and determining matching 
information for each matching portion of the recorded information and 
feature portion using a matching algorithm configured to map the visual 
feature to a portion of any of a plurality of recorded information that 
matches the feature portion; and of the recorded information determined to 
match the visual feature, 
(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. Also 
Chiu further illustrated at FIG. 2 browser interface 100a having window 105 displays the 
relative time in which video frame 200a is displayed during the course of event 50. Also 
Chiu further disclose the comparing step includes comparing color information from the 
scanned document to color information from the plurality of video frames using the 
orthonormal transformation, such as discrete cosines transform (e.g., a matching 
algorithm configured to map the visual feature to a portion of any of a plurality of 
recorded information that matches the feature portion). 
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metadata including the matching information, 

(See Fig. 2 and at Column 3 lines 62-67^ Chiu illustrated in FIG. 2, window 105 
displays the relative time in which video frame 200a is displayed during the course of 
event 50. Similarly, bar 104a represents the time and duration in which document 101 is 
presented as projection 101b in window 203. This interpretation is supported by 
Applicant's Specification, which states "metadata may indicate how long the slide was 
discussed" at Pages 14 Para [0066] Lines 11-12. ) 

(/ 103 ( - 100a 
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FIG. 2 

generating a user selectable object providing a user with access to 
the portion of the recorded information corresponding to the visual feature, 
and inserting the user selectable object into the electronic representation 
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of the document when the computer system locates a portion of the 
recorded information determined to match the visual feature, including the 
user selectable object, and the user selectable object being placed in a 
position associated with the extracted feature and allowing the user to 
access the portion of the recorded information as an embedded video link 
in the user selectable object selecting the user selectable object, the user- 
selectable object being able to access the portion of the recorded 
information using the metadata in the composite document; 
(See Fig. 3 and at Column 4 lines 4-35-> Chiu illustrated in FIG. 3, illustrated illustrates 
a browser interface 300 for accessing a segment of video file 200 which references 
scanned document 101a (e.g., user selectable object). In addition, Chiu added scanned 
documents 301 and 101a may be represented by hyperlinked universal resource 
location ("URL") addresses. The URL links a scanned document to a corresponding 
segment of video file in which the scanned document is referenced (e.g. an embedded 
video link). 

In addition Chui does not expressly teach, but Coar teaches: 

storing the composite electronic representation as a PDF, HyperText 
Transfer Language (HTML), Flash or Word formatted document for access 
by the user or another user accessing the composite electronic document. 

(@ the Abstract and @ Para [0037]-> Coar discloses the contents of the package will 
be stored unchanged from their original format (other than to use compression methods 
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to minimize the overall size of the package), in order to allow the original native 
application that created the object to again be used to interact and manipulate the 
object. Also Coar further discloses an electronic file that may be used to store any type 
of object such as an image, document, database, or any computer generated file 
including word processing, database, EDI or any other type of file. If the user so desires, 
applications may be created to convert documents into other formats such as converting 
a MS-Word document into an Adobe Acrobat (PDF file) prior to the document being 
added to the VirPack [@ Para [0069]].) 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui's method of comparing the visual feature to the 
recorded information to determine a portion of the recorded information that matches 
the visual feature and linking a video to a scan document, to include a means of said 
storing the composite electronic representation as a PDF, HyperText Transfer 
Language (HTML), Flash or Word formatted document for access by the user or 
another user accessing the composite electronic document as taught by Coar. One of 
ordinary skill in the art would have been motivated to modify this combination because 
Chui and Coar are from the same field of endeavor of generating a compound 
document (text, image, video) and provides a predictable result of generating a 
composite electronic presentation with a browser interface for accessing a segment of 
video file which references scanned document- see Chui at Fig. 3 and Col. 4, Line 1- 
15). 
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Regarding Independent claim 21, 

is fully incorporated similar subject of claim 1 cited above, and is similarly 
rejected along the same rationale. Thus, Chiu and Coar disclose every limitation 
of Claim 21 and provide proper reasons to combine, as indicated in the above 
rejection for Claim 1 . 

Regarding Independent claim 34, 

Claim 34 recites a computer program product stored on a computer- 
readable medium for implement a method recited in Claim 1 . Thus, Chiu and 
Coar disclose every limitation of Claim 34 and provide proper reason to combine, 
as indicated in the above rejections for Claim 1- Also See Chiu at Fig.1 . 

Regarding Independent claim 44, 

Claim 44 recites a computer program product stored on a computer- 
readable medium for implement a method recited in Claim 21 . Thus, Chiu and 
Coar disclose every limitation of Claim 44 and provide proper reason to combine, 
as indicated in the above rejections for Claim 21- Also @ Fig 1 of Chiu. 



Regarding Independent claim 51, 

Claim 51 recites a data processing system for implement a method recited 
in Claim 1. Thus, Chiu and Coar disclose every limitation of Claim 51 and 
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provide proper reason to combine, as indicated in the above rejections for Claim 
1- Also See Chiu at Fig. 1. 

Regarding Independent claim 61, 

Claim 61 recites a data processing system for implement a method recited 
in Claim 21 . Thus, Chiu and Coar disclose every limitation of Claim 61 and 
provide proper reason to combine, as indicated in the above rejections for Claim 
61- Also See Chiu at Fig. 1. 

Regarding Independent claim 68, 

Claim 68 recites a system for implement a method recited in Claim 1 . 
Thus, Chiu and Coar disclose every limitation of Claim 68 and provide proper 
reason to combine, as indicated in the above rejections for Claim 1- Also See 
Chiu at Fig. 1. 

Regarding Independent claim 69, 

Claim 69 recites a data processing system for implement a method recited 
in Claim 21 . Thus, Chiu and Coar disclose every limitation of Claim 69 and 
provide proper reason to combine, as indicated in the above rejections for Claim 
1- Also See Chiu at Fig. 1. 
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Regarding claims 2, 22, 35, 45, 52, and 62, 
Chiu and Coar further comprise: 

determining associating information for the recorded information 
that corresponds to the extracted feature. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. Chiu further discloses a method for linking a scanned document 
to a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step-See 
Chiu at Col. 2, Lines 15-25, discloses. 

Also see Chiu at Fig. 2, illustrates bar 104a represents the time and duration in 
which document 101 is presented as projection 101b in window 203.) 

Regarding claims 3, 23, 46 and 63, 

Chiu and Coar further comprise: 

wherein the association information comprises time information and 
source information for recorded information. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
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indexes to the video. Chiu further discloses a method for linking a scanned document 
to a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step-See 
Chiu at Col. 2, Lines 15-25, discloses. 

Also see Chiu at Fig. 2, illustrates bar 104a represents the time and duration in 
which document 101 is presented as projection 101b in window 203.) 

Regarding claims 4, 36, and 53, 

Chiu and Coar further comprise: 

associating the association information with the determined additional 
information in the composite electronic representation. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
(i.e. additional information) made on the paper documents during the meeting can be 
extracted and used as indexes to the video.) 

Regarding claims 5, 24, 37, 47, 54 and 64, 
Chiu and Coar further comprise: 

receiving a selection to the determined additional information in the 
composite electronic representation and using the association information 
for the additional information to access the recorded information. 
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(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 

Regarding claims 6, 25, 38 and 55, 

Chiu and Coar further comprise: 

accessing the recorded information using the determined additional 
information. 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 

Regarding claims 7, and 26, 

Chiu and Coar further comprise: 

displaying the accessed recorded information. 
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(See Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for accessing 
a segment of video file 200 which references scanned document 101a.) 

Regarding claims 8, and 27, 

Chiu and Coar further comprise: 

playing the accessed information. 

(See Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for accessing 
a segment of video file 200 which references scanned document 101a.) 

Regarding claims 9, 29, 39, 49, 56 and 66, 
Chiu and Coar further comprise: 

performing at least one of emailing, printing, storing, and copying 
the created composite electronic representation. 

(See Chiu at Fig. 5 and Col. 8 Line 5^discloses memory item 806 and persistent 
storage 808 stores a scanned digital document 101a and video file 200.) 

Regarding claims 11, 41 and 58, 

Chiu and Coar further comprise: 

wherein the received electronic representation of the paper 
document includes notes taken by a user, wherein the created composite 
electronic representation includes the notes taken by the user. 
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(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed. When more than one set of handouts have been annotated, these ink 
strokes may be extracted and selectively layered over the common background of the 
scanned document. Another way to display the ink annotations and notes is simply to 
show them without a background. In any case, the ink strokes may be hyperlinked to 
play their corresponding segment in the video recording.) 

Regarding claim 13, 

Chiu and Coar further comprise: 

determining a document that includes the recorded information 
using the extracted feature. 

(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed. When more than one set of handouts have been annotated, these ink 
strokes may be extracted and selectively layered over the common background of the 
scanned document. Another way to display the ink annotations and notes is simply to 
show them without a background. In any case, the ink strokes may be hyperlinked to 
play their corresponding segment in the video recording.) 
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Regarding claim 14, 

Chiu and Coar further comprise: 

determining a portion of the document that includes the information 

corresponding to the feature. 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step.) 

Regarding claims 15, 31, 42, 50 and 59, 

Chiu and Coar further comprise: 

an identifier to a location in the recorded information, wherein the 

information in the recorded information corresponding to the feature is 

determined using the identifier. 
(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step). 

Regarding claims 16 and 32, 

Chiu and Coar further comprise: 
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the identifier comprises at least one of a barcode and signature 
information. 

(See Coar at Fig. 1 and at Para 86, discloses a machine-readable symbol such as a 
high-density barcode.) 

Accordingly, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui's method, to include a means of said the identifier 
comprises at least one of a barcode and signature information as taught by Coar. One 
of ordinary skill in the art would have been motivated to modify this combination 
because Chui and Coar are from the same field of endeavor of electronic presentation 
material associated with the scan document, and provides a predictable result of 
generating a composite electronic presentation with a browser interface for accessing a 
segment of video file which references scanned document- see Chui at Fig. 3 and Col. 
4, Line 1-15.) 

Regarding claims 17-18, 

Chiu and Coar further comprise: 

receiving the electronic representation comprises receiving a scan of 
the document, the document being a paper document, wherein receiving 
the electronic representation comprises determining an electronic image of 
the document, the document being a paper document. 
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(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed.) 

Regarding claims 19, and 33, 

Chiu and Coar further comprise: 

wherein receiving the electronic representation comprises receiving 
the electronic representation in response to an input from a user indicating 
that the composite electronic representation should be created. 
(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. This interpretation is supported by Applicant's Specification, which 
states "creating a composite electronic representation comprising the determined 
information" at Pages 3 Para 15. ) 

Regarding claims 20, 30, 43, 60 and 67, 
Chiu and Coar further comprise: 

the document comprises a paper document. 

(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed.) 
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It is noted that any citations to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to 
be limiting in any way. A reference is relevant for all it contains and may be relied upon 
for all that it would have reasonably suggested to one having ordinary skill in the art. 
See, MPEP2123. 

Response to Arguments 

Applicant's arguments filed with current paper have been considered but are 
moot in view of the new ground(s) of rejections as cited above. 

This is a NonFinal office action in order to provide applicant the opportunity to 
response to the new grounds of rejection, which is set forth above. 

In addition, the Examiner maintains the Chiu and Coar reference; since 
disclosed browser interface (item 300) for accessing a segment of video file (item 
200) which references scanned document (item 101a), wherein browser interface 300 
includes window (item 302) for viewing scanned documents (item 301 and 101a). This 
allows a user may double-click on the scanned document (item 301 or 101a) and 
browser interface (item 100a) will play in window (item 203) the corresponding recorded 
segment of event (item 50) referencing the scanned documents. Also the scanned 
documents (item 301 and 101a) may be represented by hyperlinked universal resource 
location ("URL") addresses, where the video may be resided- See Chiu at Fig. 1-5 and 
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at Column 3, Line 45 through Column 4, Line 15. Also Chiu further disclosed the links 
between a scanned document and a segment of video matching the scanned document 
(col. 2, lines 15-17; abstract) also illustrated at Figs. 2-3 and 5 of Chiu. In addition, the 
examiner maintains the Coar reference; since Coar discloses the contents of the 
package will be stored unchanged from their original format (other than to use 
compression methods to minimize the overall size of the package), in order to allow the 
original native application that created the object to again be used to interact and 
manipulate the object. Also Coar further discloses an electronic file that may be used to 
store any type of object such as an image, document, database, or any computer 
generated file including word processing, database, EDI or any other type of file. If the 
user so desires, applications may be created to convert documents into other formats 
such as converting a MS-Word document into an Adobe Acrobat (PDF file) prior to the 
document being added to the VirPack [@ the Abstract and @ Para [0037] Para [0069] 
of Coar]. Also Coar further discloses a machine-readable symbol such as a high-density 
barcode of claims 16 and 32 [see Coar at Col. 7, Lines 1-10]. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Quoc A. Tran whose telephone number is 571-272- 
8664. The examiner can normally be reached on Mon through Fri 8AM - 5PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doug Hutton can be reached on (571)272-4137. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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